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Continued Examination Under 37 CFR 1.114 

1 . A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application 
is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 

1 .17(e) has been timely paid, the finality of the previous Office action has been withdrawn 
pursuant to 37 CFR 1.114. Applicant's submission filed on 07/28/2009 has been entered. 

Response to Amendment and Arguments 

2. Applicant's arguments with regard to claims 1-3, 5-11, 13-19 and 21-23 have 
been fully considered, but are not considered persuasive because of the following 
reasons: 

Regarding claim 1, the Applicant argues (page 10 of the Remarks) that the 
references do not teach the amended concept of the camera and the display of the handheld video 
phone system are integrated in a single unit and therefore would not be able to provide an image 
of a portion of a head of a user of the video phone system. The Examiner respectfully disagrees. 
Kim U.S. 2003/0064685 clearly teaches this concept at FIG. 4-5; page 1, [0012]; page 2, [0017] 
and [0021]. Thus, this would clearly allow one of ordinary skill in the art to capture an image 
that may just contains a portion of a head of a user of the video phone (or PDA). 

The Examiner believes that all the arguments of the Applicant have been properly 
addressed and explained. Thus, the rejections of all of the claims are maintained. 
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Claim Rejections - 35 USC § 103 

1 . The following is a quotation of 35 U.S. C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the im ention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the stibject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

2. Claims 1-3, 5-11, 13-19 and 21-23 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over the combination of U.S. Patent 6,806,898 by Toyama et al. ("Toyama"), U.S. 
Patent 6,707,933 by Mariani et al. ("Mariani"), Schneiderman et al. U.S. Pub. No. 
2002/0159627, and fiirther in view of Kim U.S. Pub. No. 2003/0064658. 

Regarding claim 1 , Toyama discloses a method (figure 8) for processing an image 
containing at least a portion of a head of a human in a video phone system, comprising: 

estimating an orientation of said head in said image using a pattern recognition technique 

(810); 

computing a three dimensional model of a face surface of said human using a computer 

vision technique (822, 824); and 

adjusting an orientation of said three dimensional face surface model to provide a frontal 
view (826). 

Toyama does not explicitly disclose a pattem recognition technique comprises a 

classification technique and does not appear to disclose that the image is kept unmodified when 
the orientation of the head is estimated to be frontal, as claimed. It appears that Toyama 
performs the computing and adjusting steps regardless of the orientation of the head. Also, 
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Toyama does not explicitly teach computing a three dimensional model on a face surface of said 
human using a computer vision technique based on the result of the classification technique. 

Mariani discloses a facial direction estimation system. In particular, for a video 
conferencing environment, Mariani teaches that "[u]sing the face direction estimation ... it is 
possible to enhance the quality of the transmitted images by generating a frontal view when the 
face is off-frontal ... or by unchanging the current satisfying frontal face which is displayed" 
(column 3/34-40) and fiirther teaches a pattern recognition technique (column 3, lines 63-65) 
comprises a classification technique (column 4, lines 25-28). Thus, Mariani, like Toyama, 
recognizes the need to adjust the orientation of the face in order to achieve eye-to-eye contact for 
video conferencing communications. Mariani further recognizes that when the original image of 
the face is a "satisfying frontal face" there is no need to change the orientation of the head, and 
the image can be left unchanged. Accordingly, it would have been obvious to one of ordinary 
skill in the art at the time of the invention to modify Toyama by Mariani to achieve the claimed 
invention since Mariani teaches that when the orientation of the head is estimated to be front- 
facing, there is no need to modify the image of the face since it is already in the desired 
orientation. 

Additionally, Schneiderman teaches a facial processing system (FIG. 6) wherein 
discloses computing a three dimensional model on a face surface of said human using a 
computer vision technique based on the result of the classification technique (a processing of 
three dimensional model utilizing decision rule mathematically to train classifiers) (page 4, 
[0054] and page 5, column 2). Modifying Toyama's method of processing human facial 
according to Schneiderman would be able to compute a three dimensional model on a face 
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surface of said iiuman using a computer vision technique based on the result of the classification 
technique. This would improve processing because it would improve estimation of the presence 
of the 3D object based on evaluation of the plurality of visual attributes (page 2, [0014]) and 
therefore, it would have been obvious to one of the ordinary skill in the art to modify Toyama 
according to Schneiderman. 

Toyama does not explicitly teach a concept of the camera and the display of the handheld 
video phone system are integrated in a single unit and thus would not be able to provide an 
image containing at least a portion of a head of a user of the video phone system to the image 
processor. Kim teaches a handheld video phone system (abstract) wherein further discloses a 
concept of the camera and the display of the handheld video phone system are integrated in a 
single unit (abstract; FIG. 1, 4-5) and thus would be able to provide an image containing at least 
a portion of a head of a user of the video phone system to the image processor. Modifying 
Toyama's method of processing human facial according to Kim would be able to teach a concept 
of the camera and the display of the handheld video phone system are integrated in a single unit 
and thus improve the portability. This would improve processing and therefore, it would have 
been obvious to one of the ordinary skill in the art to modify Toyama according to Kim. 

Regarding claim 2, Toyama discloses said computing act further comprises an act of 
using a symmetric face assumption to obtain a complete three dimensional face surface model 
for a profile view (824). 

Regarding claim 3, Toyama discloses said computing act further comprises an act of 
employing a structure from motion technique to obtain said three dimensional face surface model 
(column 13/1-10). 
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Regarding claim 5, Toyama discloses said computing step generates a morphable three 
dimensional model (colunm 10/60 — 1 1/43: the 3D model of the face can be morphed, i.e., 
rotated, moved, changed, etc., into a desired shape/pose). 

Regarding claim 6, Toyama discloses the step of mapping said three dimensional face 
surface model having an adjusted orientation to a two dimensional space (828). 

Regarding claim 7, Toyama discloses the step of transmitting said adjusted image to a 
remote user (see e.g. figure 2). 

Regarding claim 8, Toyama discloses the step of presenting said adjusted image to a local 
user (see e.g. figures 1 and 2). 

Regarding claim 9, Toyama discloses an image processor (102, figure 1) for use in a 
video phone system, comprising: 

a memory (104) for storing an image containing at least a portion of a head of a human; 

and 

a head pose corrector that 

(i) estimates an orientation of said head in said image using a pattern recognition 
technique (810, figure 8); 

(iii) computes a three dimensional model of a face surface of said human using a 
computer vision technique (822, 824, figure 8); and 

(iv) adjusts an orientation of said three dimensional face surface model to provide 
a fi-ontal view (826, figure 8). 
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Toyama does not appear to disclose that the image is kept unmodified when the 
orientation of the head is estimated to be frontal, as claimed. It appears that Toyama performs the 
computing and adjusting steps regardless of the orientation of the head. 

Mariani discloses a facial direction estimation system. In particular, for a video 
conferencing environment, Mariani teaches that "[u]sing the face direction estimation ... it is 
possible to enhance the quality of the transmitted images by generating a frontal view when the 
face is off-frontal ... or by unchanging the current satisfying frontal face which is displayed" 
(column 3/34-40). Thus, Mariani, like Toyama, recognizes the need to adjust the orientation of 
the face in order to achieve eye-to-eye contact for video conferencing communications. Mariani 
further recognizes that when the original image of the face is a "satisfying frontal face" there is 
no need to change the orientation of the head, and the image can be left unchanged. Accordingly, 
it would have been obvious to one of ordinary skill in the art at the time of the invention to 
modify Toyama by Mariani to achieve the claimed invention since Mariani teaches that when the 
orientation of the head is estimated to be front-facing, there is no need to modify the image of the 
face since it is already in the desired orientation. 

Toyama does not explicitly teach a concept of the camera and the display of the handheld 
video phone system are integrated in a single unit and thus would not be able to provide an 
image containing at least a portion of a head of a user of the video phone system to the image 
processor. Kim teaches a handheld video phone system (abstract) wherein further discloses a 
concept of the camera and the display of the handheld video phone system are integrated in a 
single unit (abstract; FIG. 1, 4-5) and thus would be able to provide an image containing at least 
a portion of a head of a user of the video phone system to the image processor. Modifying 
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Toyama's method of processing human facial according to Kim would be able to teach a concept 
of the camera and the display of the handheld video phone system are integrated in a single unit 
and thus improve the portability. This would improve processing and therefore, it would have 
been obvious to one of the ordinary skill in the art to modify Toyama according to Kim. 

Regarding claim 10, Toyama discloses said head pose corrector is further configured to 
use a symmetric face assumption to obtain a complete three dimensional face surface model for a 
profile view (824, figure 8). 

Regarding claim 11, Toyama discloses said head pose corrector is further configured to 
employ a structure from motion technique to obtain said three dimensional face surface model 
(column 13/1-10). 

Regarding claim 13, Toyama discloses said three dimensional face surface model is a 
morphable three dimensional model (column 10/60 — 1 1/43: the 3D model of the face can be 
morphed, i.e., rotated, moved, changed, etc., into a desired shape/pose). 

Regarding claim 14, Toyama discloses said head pose corrector is fiirther configured to 
map said three dimensional face surface model having an adjusted orientation to a two 
dimensional modified image (828, figure 8). 

Regarding claim 15, Toyama discloses said two dimensional modified image is 
transmitted to a remote user (see e.g. figure 2). 

Regarding claim 16, Toyama discloses said two dimensional modified image is presented 
to a local user (see e.g. figures 1 and 2). 

Regarding claim 17, Toyama discloses a video phone system (figures 1-2), comprising: 
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a memory (104, figure 1) for storing an image containing at least a portion of a head of a 
human; and 

a head pose corrector (102, figure 1) that 

(i) estimates an orientation of said head in said image using a pattern recognition 
technique (810, figure 8); 

(iii) computes a three dimensional model of a face surface of said human using a 
computer vision technique (822, 824, figure 8); and 

(iv) adjusts an orientation of said three dimensional face surface model to provide 
a fi-ontal view (826, figure 8). 

Toyama does not appear to disclose that the image is kept unmodified when the 
orientation of the head is estimated to be fi-ontal, as claimed. It appears that Toyama performs the 
computing and adjusting steps regardless of the orientation of the head. 

Mariani discloses a facial direction estimation system. In particular, for a video 
conferencing environment, Mariani teaches that "[ujsing the face direction estimation ... it is 
possible to enhance the quality of the transmitted images by generating a frontal view when the 
face is ofif-fi'ontal ... or by unchanging the current satisfying frontal face which is displayed" 
(column 3/34-40). Thus, Mariani, like Toyama, recognizes the need to adjust the orientation of 
the face in order to achieve eye-to-eye contact for video conferencing communications. Mariani 
further recognizes that when the original image of the face is a "satisfying frontal face" there is 
no need to change the orientation of the head, and the image can be left unchanged. Accordingly, 
it would have been obvious to one of ordinary skill in the art at the time of the invention to 
modify Toyama by Mariani to achieve the claimed invention since Mariani teaches that when the 
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orientation of the head is estimated to be front-facing, there is no need to modify the image of the 
face since it is already in the desired orientation. 

Toyama does not explicitly teach a concept of the camera and the display of the handheld 
video phone system are integrated in a single unit and thus would not be able to provide an 
image containing at least a portion of a head of a user of the video phone system to the image 
processor. Kim teaches a handheld video phone system (abstract) wherein further discloses a 
concept of the camera and the display of the handheld video phone system are integrated in a 
single unit (abstract; FIG. 1, 4-5) and thus would be able to provide an image containing at least 
a portion of a head of a user of the video phone system to the image processor. Modifying 
Toyama's method of processing human facial according to Kim would be able to teach a concept 
of the camera and the display of the handheld video phone system are integrated in a single unit 
and thus improve the portability. This would improve processing and therefore, it would have 
been obvious to one of the ordinary skill in the art to modify Toyama according to Kim. 

Regarding claim 18, Toyama discloses said head pose corrector is fiirther configured to 
use a symmetric face assumption to obtain a complete three dimensional face surface model for a 
profile view (824, figure 8). 

Regarding claim 19, Toyama discloses said head pose corrector is further configured to 
employ a structure from motion technique to obtain said three dimensional face surface model 
(column 13/1-10). 
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Regarding claim 21, Toyama discloses said head pose corrector is fiirther configured to 
map said three dimensional face surface model having an adjusted orientation to a two 
dimensional modified image (828, figure 8). 

Regarding claim 22, Toyama discloses said two dimensional modified image is 
transmitted to a remote user (see e.g. figure 2). 

Regarding claim 23, Toyama discloses said two dimensional modified image is presented 
to a local user (see e.g. figures 1 and 2). 
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Contact Information 

1 . Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to BRIAN Q. LE whose telephone number is (571)272-7424. The 
examiner can normally be reached on 8:30 A.M - 5:30 P.M. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Samir Ahmed can be reached on 571-272-7413. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an apphcation may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/Brian Q Le/ 

Primary Examiner, Art Unit 2624 
10/29/09 



